feat(ai-grok): video generation adapter for the grok-imagine video models#742
Open
tombeckenham wants to merge 2 commits into
Open
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds Grok Imagine video generation: model metadata and provider option types, a GrokVideoAdapter implementing create/poll/getUrl flows (with usage/cost mapping), factories and exports, tests, docs, and example app wiring. ChangesGrok Imagine Video Adapter
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🚀 Changeset Version Preview2 package(s) bumped directly, 0 bumped as dependents. 🟨 Minor bumps
🟩 Patch bumps
|
|
View your CI Pipeline Execution ↗ for commit 3d09c62
☁️ Nx Cloud last updated this comment at |
@tanstack/ai
@tanstack/ai-angular
@tanstack/ai-anthropic
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-mcp
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/media/video-generation.md (1)
315-315:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winRemove
as anytype assertion from documentation code sample.The coding guidelines explicitly prohibit
astype-assertion casts in documentation code samples (exceptas const). Examples must type-check without type casts.Consider properly typing the
sizeparameter in the input validator or using a type guard instead:.inputValidator((data: { prompt: string; size?: '1280x720' | '720x1280' | '1792x1024' | '1024x1792'; duration?: number }) => data)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/media/video-generation.md` at line 315, Remove the "as any" cast on the size assignment in the docs sample and ensure the sample types-check without casts: update the .inputValidator signature to type the incoming data.size as the allowed union ('1280x720' | '720x1280' | '1792x1024' | '1024x1792') or add a short type guard that validates/normalizes data.size before assigning to size; reference the .inputValidator callback and the size property so the sample compiles without using "as any".Source: Coding guidelines
🧹 Nitpick comments (1)
packages/ai-grok/tests/video-adapter.test.ts (1)
1-1: ⚡ Quick winPlace this unit test alongside the source file per repo rule.
video-adapter.test.tscurrently lives underpackages/ai-grok/tests/, but the guideline requires*.test.tsfiles to be colocated with source. Please move it next to the adapter/provider source it validates (for example underpackages/ai-grok/src/...).As per coding guidelines, "
**/*.test.ts: Place unit tests alongside source code in*.test.tsfiles".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-grok/tests/video-adapter.test.ts` at line 1, The test file video-adapter.test.ts must be moved from packages/ai-grok/tests/ to be colocated with the adapter/provider source it validates (e.g., under packages/ai-grok/src/ alongside the video adapter implementation), update any relative import paths in video-adapter.test.ts to reflect the new location, and adjust any test-runner or build config references if they rely on the old tests/ directory; locate the adapter/provider source by name (video-adapter, VideoAdapter, or similar) and place the test file next to that module so imports and module resolution remain correct.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/ai-grok/src/adapters/video.ts`:
- Around line 151-208: The validation calls (validateVideoSize,
validateVideoDuration) in createVideoJob can throw before the existing
try/catch, so wrap those validations and the rest of the function body inside
the same try block (or add an outer try that encloses validations) so any thrown
validation errors are caught; in the catch call logger.errors with
toRunErrorPayload and source `${this.name}.createVideoJob` (same shape as
current catch) and rethrow the error to preserve behavior.
---
Outside diff comments:
In `@docs/media/video-generation.md`:
- Line 315: Remove the "as any" cast on the size assignment in the docs sample
and ensure the sample types-check without casts: update the .inputValidator
signature to type the incoming data.size as the allowed union ('1280x720' |
'720x1280' | '1792x1024' | '1024x1792') or add a short type guard that
validates/normalizes data.size before assigning to size; reference the
.inputValidator callback and the size property so the sample compiles without
using "as any".
---
Nitpick comments:
In `@packages/ai-grok/tests/video-adapter.test.ts`:
- Line 1: The test file video-adapter.test.ts must be moved from
packages/ai-grok/tests/ to be colocated with the adapter/provider source it
validates (e.g., under packages/ai-grok/src/ alongside the video adapter
implementation), update any relative import paths in video-adapter.test.ts to
reflect the new location, and adjust any test-runner or build config references
if they rely on the old tests/ directory; locate the adapter/provider source by
name (video-adapter, VideoAdapter, or similar) and place the test file next to
that module so imports and module resolution remain correct.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: b6eef03c-6d23-44d3-b96b-75cc68878c1b
📒 Files selected for processing (10)
.changeset/grok-imagine-video-adapter.mddocs/adapters/grok.mddocs/config.jsondocs/media/video-generation.mdpackages/ai-grok/src/adapters/video.tspackages/ai-grok/src/index.tspackages/ai-grok/src/model-meta.tspackages/ai-grok/src/video/video-provider-options.tspackages/ai-grok/tests/video-adapter.test.tspackages/ai/skills/ai-core/media-generation/SKILL.md
Comment on lines
+151
to
+208
| async createVideoJob( | ||
| options: VideoGenerationOptions<GrokVideoProviderOptions>, | ||
| ): Promise<VideoJobResult> { | ||
| const { model, prompt, size, modelOptions, logger } = options | ||
|
|
||
| validateVideoSize(model, size) | ||
| validateVideoDuration(model, options.duration) | ||
| validateVideoDuration(model, modelOptions?.duration) | ||
| const duration = options.duration ?? modelOptions?.duration | ||
|
|
||
| // The generic `size` option carries an "aspectRatio_resolution" template | ||
| // (e.g. '16:9_720p') and maps to the Imagine API's `aspect_ratio` / | ||
| // `resolution` parameters; explicit modelOptions win over the template. | ||
| const parsedSize = size !== undefined ? parseGrokVideoSize(size) : undefined | ||
| const request = { | ||
| model, | ||
| prompt, | ||
| ...(parsedSize && { | ||
| aspect_ratio: parsedSize.aspectRatio, | ||
| ...(parsedSize.resolution !== undefined && { | ||
| resolution: parsedSize.resolution, | ||
| }), | ||
| }), | ||
| ...(duration !== undefined && { duration }), | ||
| ...modelOptions, | ||
| } | ||
|
|
||
| try { | ||
| logger.request( | ||
| `activity=video.create provider=${this.name} model=${model} size=${size ?? 'default'} duration=${duration ?? 'default'}`, | ||
| { provider: this.name, model }, | ||
| ) | ||
|
|
||
| const response = await this.request('/videos/generations', { | ||
| method: 'POST', | ||
| body: JSON.stringify(request), | ||
| }) | ||
| if (!response.ok) { | ||
| throw new Error( | ||
| `grok: video generation request failed (${response.status} ${response.statusText}): ${await this.errorMessage(response)}`, | ||
| ) | ||
| } | ||
|
|
||
| const result = (await response.json()) as GrokVideoCreateResponse | ||
| if (!result.request_id) { | ||
| throw new Error( | ||
| 'grok: video generation response contained no request_id', | ||
| ) | ||
| } | ||
| return { jobId: result.request_id, model } | ||
| } catch (error: unknown) { | ||
| logger.errors(`${this.name}.createVideoJob fatal`, { | ||
| error: toRunErrorPayload(error, `${this.name}.createVideoJob failed`), | ||
| source: `${this.name}.createVideoJob`, | ||
| }) | ||
| throw error | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
Verify that validation errors before the API call are properly logged.
The validation calls validateVideoSize and validateVideoDuration (lines 156-158) can throw errors before the try block starts at line 178. If validation fails, the error bypasses the catch block's logger, so validation failures won't be logged with the structured error format.
Consider wrapping the entire function body in the try-catch or adding a separate catch wrapper:
🛡️ Proposed fix to ensure validation errors are logged
async createVideoJob(
options: VideoGenerationOptions<GrokVideoProviderOptions>,
): Promise<VideoJobResult> {
+ try {
const { model, prompt, size, modelOptions, logger } = options
validateVideoSize(model, size)
validateVideoDuration(model, options.duration)
validateVideoDuration(model, modelOptions?.duration)
const duration = options.duration ?? modelOptions?.duration
// The generic `size` option carries an "aspectRatio_resolution" template
// (e.g. '16:9_720p') and maps to the Imagine API's `aspect_ratio` /
// `resolution` parameters; explicit modelOptions win over the template.
const parsedSize = size !== undefined ? parseGrokVideoSize(size) : undefined
const request = {
model,
prompt,
...(parsedSize && {
aspect_ratio: parsedSize.aspectRatio,
...(parsedSize.resolution !== undefined && {
resolution: parsedSize.resolution,
}),
}),
...(duration !== undefined && { duration }),
...modelOptions,
}
- try {
logger.request(
`activity=video.create provider=${this.name} model=${model} size=${size ?? 'default'} duration=${duration ?? 'default'}`,
{ provider: this.name, model },
)
const response = await this.request('/videos/generations', {
method: 'POST',
body: JSON.stringify(request),
})
if (!response.ok) {
throw new Error(
`grok: video generation request failed (${response.status} ${response.statusText}): ${await this.errorMessage(response)}`,
)
}
const result = (await response.json()) as GrokVideoCreateResponse
if (!result.request_id) {
throw new Error(
'grok: video generation response contained no request_id',
)
}
return { jobId: result.request_id, model }
- } catch (error: unknown) {
+ } catch (error: unknown) {
logger.errors(`${this.name}.createVideoJob fatal`, {
error: toRunErrorPayload(error, `${this.name}.createVideoJob failed`),
source: `${this.name}.createVideoJob`,
})
throw error
- }
+ }
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| async createVideoJob( | |
| options: VideoGenerationOptions<GrokVideoProviderOptions>, | |
| ): Promise<VideoJobResult> { | |
| const { model, prompt, size, modelOptions, logger } = options | |
| validateVideoSize(model, size) | |
| validateVideoDuration(model, options.duration) | |
| validateVideoDuration(model, modelOptions?.duration) | |
| const duration = options.duration ?? modelOptions?.duration | |
| // The generic `size` option carries an "aspectRatio_resolution" template | |
| // (e.g. '16:9_720p') and maps to the Imagine API's `aspect_ratio` / | |
| // `resolution` parameters; explicit modelOptions win over the template. | |
| const parsedSize = size !== undefined ? parseGrokVideoSize(size) : undefined | |
| const request = { | |
| model, | |
| prompt, | |
| ...(parsedSize && { | |
| aspect_ratio: parsedSize.aspectRatio, | |
| ...(parsedSize.resolution !== undefined && { | |
| resolution: parsedSize.resolution, | |
| }), | |
| }), | |
| ...(duration !== undefined && { duration }), | |
| ...modelOptions, | |
| } | |
| try { | |
| logger.request( | |
| `activity=video.create provider=${this.name} model=${model} size=${size ?? 'default'} duration=${duration ?? 'default'}`, | |
| { provider: this.name, model }, | |
| ) | |
| const response = await this.request('/videos/generations', { | |
| method: 'POST', | |
| body: JSON.stringify(request), | |
| }) | |
| if (!response.ok) { | |
| throw new Error( | |
| `grok: video generation request failed (${response.status} ${response.statusText}): ${await this.errorMessage(response)}`, | |
| ) | |
| } | |
| const result = (await response.json()) as GrokVideoCreateResponse | |
| if (!result.request_id) { | |
| throw new Error( | |
| 'grok: video generation response contained no request_id', | |
| ) | |
| } | |
| return { jobId: result.request_id, model } | |
| } catch (error: unknown) { | |
| logger.errors(`${this.name}.createVideoJob fatal`, { | |
| error: toRunErrorPayload(error, `${this.name}.createVideoJob failed`), | |
| source: `${this.name}.createVideoJob`, | |
| }) | |
| throw error | |
| } | |
| } | |
| async createVideoJob( | |
| options: VideoGenerationOptions<GrokVideoProviderOptions>, | |
| ): Promise<VideoJobResult> { | |
| try { | |
| const { model, prompt, size, modelOptions, logger } = options | |
| validateVideoSize(model, size) | |
| validateVideoDuration(model, options.duration) | |
| validateVideoDuration(model, modelOptions?.duration) | |
| const duration = options.duration ?? modelOptions?.duration | |
| // The generic `size` option carries an "aspectRatio_resolution" template | |
| // (e.g. '16:9_720p') and maps to the Imagine API's `aspect_ratio` / | |
| // `resolution` parameters; explicit modelOptions win over the template. | |
| const parsedSize = size !== undefined ? parseGrokVideoSize(size) : undefined | |
| const request = { | |
| model, | |
| prompt, | |
| ...(parsedSize && { | |
| aspect_ratio: parsedSize.aspectRatio, | |
| ...(parsedSize.resolution !== undefined && { | |
| resolution: parsedSize.resolution, | |
| }), | |
| }), | |
| ...(duration !== undefined && { duration }), | |
| ...modelOptions, | |
| } | |
| logger.request( | |
| `activity=video.create provider=${this.name} model=${model} size=${size ?? 'default'} duration=${duration ?? 'default'}`, | |
| { provider: this.name, model }, | |
| ) | |
| const response = await this.request('/videos/generations', { | |
| method: 'POST', | |
| body: JSON.stringify(request), | |
| }) | |
| if (!response.ok) { | |
| throw new Error( | |
| `grok: video generation request failed (${response.status} ${response.statusText}): ${await this.errorMessage(response)}`, | |
| ) | |
| } | |
| const result = (await response.json()) as GrokVideoCreateResponse | |
| if (!result.request_id) { | |
| throw new Error( | |
| 'grok: video generation response contained no request_id', | |
| ) | |
| } | |
| return { jobId: result.request_id, model } | |
| } catch (error: unknown) { | |
| logger.errors(`${this.name}.createVideoJob fatal`, { | |
| error: toRunErrorPayload(error, `${this.name}.createVideoJob failed`), | |
| source: `${this.name}.createVideoJob`, | |
| }) | |
| throw error | |
| } | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-grok/src/adapters/video.ts` around lines 151 - 208, The
validation calls (validateVideoSize, validateVideoDuration) in createVideoJob
can throw before the existing try/catch, so wrap those validations and the rest
of the function body inside the same try block (or add an outer try that
encloses validations) so any thrown validation errors are caught; in the catch
call logger.errors with toRunErrorPayload and source
`${this.name}.createVideoJob` (same shape as current catch) and rethrow the
error to preserve behavior.
AlemTuzlak
approved these changes
Jun 19, 2026
tombeckenham
force-pushed
the
705-featai-grok-video-generation-adapter-for-the-grok-imagine-video-models
branch
from
c6e1573 to
1bfbe33
Compare
tombeckenham
commented
Jun 23, 2026
tombeckenham
left a comment
tombeckenham
left a comment
Contributor
Author
There was a problem hiding this comment.
Hold off on this, there have been some changes on the model, and the example doesn't work
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
packages/ai/skills/ai-core/media-generation/SKILL.md (1)
457-462: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winClarify grokVideo image-to-video API usage.
Lines 459–460 describe image-to-video as "via an
imageprompt part as the starting frame," but the PR objectives specify the API contract usesmodelOptions.image: { url }, not prompt parts. Align the documentation with the actual adapter implementation to prevent user confusion.For consistency with the Veo example above (lines 440–455), consider adding a brief grokVideo code snippet showing the exact
modelOptions.imagesyntax and how it differs from the generic prompt-part pattern.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai/skills/ai-core/media-generation/SKILL.md` around lines 457 - 462, The grokVideo adapter documentation incorrectly states it uses "an image prompt part as the starting frame" when the actual implementation uses modelOptions.image: { url } instead. In the grokVideo description around lines 459-460, replace the mention of image prompt parts with accurate documentation specifying the modelOptions.image syntax with the url property. Additionally, add a brief code snippet example for grokVideo that demonstrates the modelOptions.image usage pattern, following the same format and structure as the Veo example shown in lines 440-455 above it, to provide users with clear guidance on how to use this feature correctly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/ts-react-media/src/components/ImageGenerator.tsx`:
- Line 30: The helper text or copy related to reference image support (located
around lines 165-171 in the ImageGenerator component) currently only mentions
Gemini as supporting reference images, which is now inaccurate. Update the
reference image support text to explicitly mention that both Gemini and xAI
direct models support this feature, ensuring users understand all available
options for models that accept reference images.
---
Nitpick comments:
In `@packages/ai/skills/ai-core/media-generation/SKILL.md`:
- Around line 457-462: The grokVideo adapter documentation incorrectly states it
uses "an image prompt part as the starting frame" when the actual implementation
uses modelOptions.image: { url } instead. In the grokVideo description around
lines 459-460, replace the mention of image prompt parts with accurate
documentation specifying the modelOptions.image syntax with the url property.
Additionally, add a brief code snippet example for grokVideo that demonstrates
the modelOptions.image usage pattern, following the same format and structure as
the Veo example shown in lines 440-455 above it, to provide users with clear
guidance on how to use this feature correctly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 36f5d04d-fc22-432d-95c3-88502f2db174
📒 Files selected for processing (12)
.changeset/grok-imagine-video-adapter.mddocs/adapters/grok.mddocs/media/video-generation.mdexamples/ts-react-media/src/components/ImageGenerator.tsxexamples/ts-react-media/src/components/VideoGenerator.tsxexamples/ts-react-media/src/lib/models.tsexamples/ts-react-media/src/lib/server-functions.tspackages/ai-grok/src/adapters/video.tspackages/ai-grok/src/model-meta.tspackages/ai-grok/src/video/video-provider-options.tspackages/ai-grok/tests/video-adapter.test.tspackages/ai/skills/ai-core/media-generation/SKILL.md
✅ Files skipped from review due to trivial changes (3)
- .changeset/grok-imagine-video-adapter.md
- docs/media/video-generation.md
- docs/adapters/grok.md
🚧 Files skipped from review as they are similar to previous changes (3)
- examples/ts-react-media/src/components/VideoGenerator.tsx
- packages/ai-grok/src/video/video-provider-options.ts
- packages/ai-grok/src/adapters/video.ts
|
|
||
| const falModels = IMAGE_MODELS.filter((m) => m.provider === 'fal') | ||
| const geminiModels = IMAGE_MODELS.filter((m) => m.provider === 'gemini') | ||
| const xaiModels = IMAGE_MODELS.filter((m) => m.provider === 'xai') |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Update the reference-image support copy to include xAI direct models.
The new xAI direct model group makes the helper text inaccurate (it currently implies only Gemini supports reference images), which can confuse users.
Suggested fix
- Supported by Gemini multimodal models only
- (gemini-3.1-flash-image-preview, gemini-3-pro-image-preview)
+ Supported by Gemini and xAI direct multimodal models
+ (gemini-3.1-flash-image-preview, gemini-3-pro-image-preview,
+ grok-imagine-image, grok-imagine-image-quality)Also applies to: 165-171
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@examples/ts-react-media/src/components/ImageGenerator.tsx` at line 30, The
helper text or copy related to reference image support (located around lines
165-171 in the ImageGenerator component) currently only mentions Gemini as
supporting reference images, which is now inaccurate. Update the reference image
support text to explicitly mention that both Gemini and xAI direct models
support this feature, ensuring users understand all available options for models
that accept reference images.
1cf70d0 to
2d85506
Compare
Contributor
Author
Note: model capabilities (text-to-video vs image-to-video)Verified against the live xAI Imagine API (and corroborated by fal's catalog, which only exposes
xAI's docs claim 1.5 supports text-to-video, but the live API returns Code that surfaces this: The image-to-video-only set lives in |
tombeckenham
commented
Jun 24, 2026
tombeckenham
left a comment
tombeckenham
left a comment
Contributor
Author
There was a problem hiding this comment.
I've read the docs again, and it does not appear that xAi does claim that grok imagine video 1.5 supports text to video. I have kept grok imagine 1.0 as the text to video model, and included 1.5 only for image to video
tombeckenham
commented
Jun 24, 2026
tombeckenham
left a comment
tombeckenham
left a comment
Contributor
Author
There was a problem hiding this comment.
Added support for durations and snapDuration
Add a grokVideo adapter for xAI's Imagine video models via the experimental
generateVideo() jobs/polling architecture (createVideoJob posts to
/v1/videos/generations, polling reads /v1/videos/{request_id}), with hosted
video URL plus usage (unitsBilled seconds + exact cost in USD).
Two models:
- grok-imagine-video (v1.0): text-to-video and image-to-video, $0.05/s.
- grok-imagine-video-1.5: image-to-video only, $0.08/s. xAI's API rejects
text-to-video on 1.5, so the adapter fails fast with a clear error telling
the caller to add a starting-frame image or use grok-imagine-video.
Image-to-video starting frames are supplied as an `image` prompt part
(resolveMediaPrompt convention; public URL or base64 data source). Adds native
xAI Direct grok image (grok-imagine-image / -quality) and video entries to the
ts-react-media example, plus docs, changeset, and the media skill.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the throwing `validateVideoDuration` with the standard duration-options mechanism. Both grok-imagine video models declare a continuous 1–15 integer- second range via a `GROK_VIDEO_DURATIONS` table, and the adapter overrides `availableDurations()` / `snapDuration()` (backed by the shared `snapToDurationOption` helper) so consumers can discover and pre-snap durations. `createVideoJob` now snaps the requested duration into range (clamp + round) instead of rejecting it, and the snapped value is spread after `...modelOptions` so it is authoritative. Adds the per-model `GrokVideoModelDurationByName` generic, narrows the `createVideoJob` signature to carry the size/duration type params, exports the new helpers/type, and documents the range in the media docs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tombeckenham
force-pushed
the
705-featai-grok-video-generation-adapter-for-the-grok-imagine-video-models
branch
from
1720516 to
3d09c62
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Changes
Adds a
grokVideoadapter to@tanstack/ai-grokfor xAI's Imagine video models, closing #705:grok-imagine-video($0.05/s) andgrok-imagine-video-1.5-preview($0.08/s), added tomodel-meta.tswithGROK_VIDEO_MODELS/GrokVideoModelexports.GrokVideoAdapter(+grokVideo/createGrokVideofactories) extendsBaseVideoAdapterwith the jobs/polling pattern —createVideoJobposts to/v1/videos/generations,getVideoStatus/getVideoUrlread/v1/videos/{request_id}. The Imagine video endpoints are plain JSON and not part of the OpenAI SDK surface, so the adapter issues direct requests with an injectablefetchseam (no global stubs needed in tests).sizetemplate consistent with the grok-imagine image models —'16:9_720p'→aspect_ratio/resolution. Video accepts1:1 | 16:9 | 9:16 | 4:3 | 3:4 | 3:2 | 2:3and480p | 720p | 1080p(a narrower set than the image models — verified against the live API, which rejects phone ratios andauto).modelOptions.image: { url }passes a starting frame (public URL or base64 data URI).usage.unitsBilled(billed seconds) andusage.cost(exact USD from the API'scost_in_usd_ticks; 10^10 ticks per dollar).examples/ts-react-mediagains "xAI Direct" text-to-video and image-to-video entries using the nativegrokVideoadapter (alongside the existing fal-hosted grok-imagine entries), with exact USD cost shown on completion.docs/adapters/grok.md,docs/media/video-generation.md(+config.jsonupdatedAt), and themedia-generationagent skill.The issue assumed the Imagine plumbing from #624 (issue #618 ) had landed; that PR is still open, so this is built directly against
mainand is independently mergeable. The coreimageInputs/metadata.roleflow from #618/#624 isn't available onmainyet — image-to-video is exposed through typedmodelOptions.imagefor now. If #624 merges before this, this PR can be rebased and theimageInputs/ media-prompt flow wired into the adapter here before merging. Otherwise this can be merged as is, and 624 could be updated to rewire this adapterTesting
packages/ai-grok/tests/video-adapter.test.ts, 27 tests) cover request shape, size-template mapping, validation, status mapping, usage, and error paths via the injected fetch seam. aimock doesn't mock the Imagine video endpoints, so coverage is unit-test based (same approach the issue calls out forimage-to-image/image-to-video).generateVideo()/getVideoJobStatus(): text-to-video (16:9_480p, 1s,unitsBilled: 1,cost: 0.05) and image-to-video viamodelOptions.image(cost: 0.052— the API bills slightly more with an image input, which is why cost is read from the API rather than computed).✅ Checklist
pnpm run test:pr.🚀 Release Impact
🤖 Generated with Claude Code
Summary by CodeRabbit
Summary